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1.0  Introduction 


Military  human  resource  planners  recognize  the  significance  of  the 
rapidly  declining  youth  manpower  pool  through  1992.  This  declining  segment  of 
the  population  and  the  increased  scrutiny  of  large  recruiting  resource  budgets 
by  Congress  and  others  has  prompted  the  services  to  search  for  methods  to  aid 
decision  makers  in  planning  and  allocating  resources  in  the  best  possible 
manner.  The  Center  for  Cybernetic  Studies  at  The  University  of  Texas  at  Austin 
has  responded  to  this  need  with  new  and  improved  methods  for  use  by  the 
services  in  manpower  planning.  Data  Envelopment  Analysis  (DEA)  represents 
one  such  method  which  has  been  developed  by  the  Center  for  Cybernetic 
Studies  and  greatly  enhanced  through  research  conducted  jointly  with  the  U.S. 
Army  Recruiting  Command  (USAREC)  since  1980.  DEA  represents  an 
important  development  in  its  own  right,  and  it  can  also  be  combined  with  other 
tools  such  as  "goal  programming"  (as  also  developed  by  Center  staff)  for  still 
further  uses  in  manpower  planning. 


2.0  Data  Envelopment  Analysis 

DEA  is  a  relatively  new  approach  developed  by  A.  Charnes,  W.  W. 
Cooper  and  others  associated  with  the  Center  for  Cybernetic  Studies,  which 
can  be  used  to  evaluate  the  efficiency  of  not-for-profit  entities  engaged  in 
operations  which  characteristically  use  multiple  inputs  to  produce  multiple 
outputs.  Although  these  activities  do  not  occur  in  markets  where  they  can  be 
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"priced”  or  otherwise  weighted,  DEA  does  not  requiie  recourse  to  preassigned  ,x 
weights  or  the  specification  of  functional  relations  between  outputs  and  inputs.  .d 

1  C 

It  requires  only  identification  of  those  outputs  and  inputs  which  are  important  to  ~ 
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the  operations  of  the  organization.  It  also  requires  identifying  the  organization  ~ 
entities  which  are  to  be  regarded  as  being  responsible  for  converting  inputs  into  lly  C*<5^ 
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outputs.  For  purposes  of  the  present  discussion,  these  terms  may  be  defined 
and  illustrated  as  follows: 

Outputs:  The  desired  outcomes  of  goods  or  services  that  an 

organization  produces.  Examples  for  USAREC  are  GSMA  contracts  and  GSMA 
applicants  as  well  as  less  tangible  items  such  as  changes  in  "propensities"  to 
enlist. 

Inputs:  Resources  utilized  to  produce  the  desired  outputs.  For 

USAREC  these  would  be  recruiters,  local  advertising  funds,  market  size, 
unemployment  and  other  pertinent  demographic  and  economic  characteristics. 

Decision  Making  Unit  (DMU^:  An  organizational  unit  (in  this  case,  a  U.S. 
Army  Recruiting  battalion  or  brigade)  which  is  charged  with  responsibility  for 
converting  inputs  into  outputs. 

Further  breakdowns  and  refinements  are  possible.  For  instance,  inputs 
may  also  be  classified  as  "discretionary"  if  they  can  be  varied  by  management, 
(as  in  the  case  of  recruiters  and  advertising  expenditures)  or  inputs  may  be 
"non-discretionary"  if  they  cannot  be  varied  by  management,  (as  is  the  case  for 
unemployment  rates).  Note  that  the  latter  constitute  important  inputs  which 
should  enter  into  the  evaluations  of  how  well  a  DMU  is  performing  in  the  outputs 
it  produces.  See  [1].  As  in  all  past  research,  the  choice  of  DMUs,  inputs  and 
outputs,  is  best  developed  in  a  collaborative  "team  effort"  by  USAREC  and  staff 
from  the  Center  for  Cybernetic  Studies. 


»  • 


The  following  figure  can  help  to  conceptualize  what  is  involved: 


Inputs 


Outputs 


Figure  2.1 


To  be  noted  is  that  the  DMU  operates  as  a  "black  box"  in  which  inputs  are 
converted  into  outputs,  and  explicit  formulation  of  the  connecting  mathematical 
relations  between  inputs  and  outputs  is  not  required.  Actual  managerial  data 
for  inputs  and  ouputs  is  utilized  to  obtain  an  efficiency  evaluation  for  each  DMU 
or  battalion.. 

A  simple  graphical  representation  of  what  is  involved  in  such  an 
efficiency  evaluation  is  provided  by  Figure  2.2. 


OUTPUT 


Figure  2.2 
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Only  one  output  and  one  input  are  used  in  this  simple  example.  As  can  be 
seen,  DMU  2  is  relatively  more  efficient  than  DMU  I,  because  it  has  produced 
more  output  with  the  same  amount  of  input  as  was  utilized  by  DMU  I.  Similarly, 
DMU  3  is  relatively  more  efficient  than  DMU  4  because  it  secured  the  same 
level  of  output  with  less  input.  Extensions  to  multiple  output-multiple  input 
situations  are  desired  for  such  efficiency  evaluations,  of  course,  and  this,  too,  is 
attended  to  by  DEA  in  ways  that  extend  beyond  the  pairwise  comparisons  used 
in  Figure  2.2. 

Efficiency  as  a  science  or  engineering  concept  is  usually  defined  in  terms 
of  an  output  to  input  ratio  in  which  the  output  and  the  input  are  measured  in  the 
same  units  with,  in  general,  output/input  <  I.  This  is  not  immediately  suitable  for 
use  when  multiple  outputs  and  multiple  inputs  need  to  be  considered  in 
possibly  different  units  of  measurement.  Hence,  DEA  extends  this  ratio  concept 
by  defining  a  "virtual  output"  to  "virtual  input"  ratio  in  which  the  outputs  and  the 
inputs  are  combined  first  into  single  virtual  outputs  and  inputs.  These  virtual 
outputs  and  virtual  inputs  are  evaluated  in  a  way  that  maximizes  the  efficiency 
score  of  each  DMU  under  consideration  (e.g.,  a  battalion)  by  reference  to  the 
evidence  on  the  input  to  output  attainments  reported  for  all  other  DMUs 
(battalions).  Efficiency  then  is  defined  for  this  application  as  below: 


COMBINED  RCTNG  OUTPUTS 
COMBINED  RCTNG  INPUTS 


EFFICIENCY 


As  was  true  for  the  single  output-to-single  input  case,  the  maximum  attainable 
efficiency  rating  is  unity  (or  100%)  and  the  theory  underlying  DEA  guarantees 
that  the  resulting  efficiency  ratings  for  each  DMU  do  not  depend  on  the  units  of 
measurement  employed. 
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The  theory  behind  this  development  as  well  as  its  methods  of 
implementation  are  documented  in  full  detail  in  [1]  and  [2]  and  hence,  need  not 
be  repeated  here.  Instead  we  shall  focus  on  some  of  the  further  developments 
that  have  now  occurred  in  response  to  the  service  needs  in  the  collaborative 
efforts  that  have  already  been  undertaken  by  CCS  and  USAREC. 

3.0  Past  Research  -  the  Ad  Mix  Experiment 

In  one  part  of  its  efforts,  the  Center  for  Cybernetic  Studies 
undertook  a  detailed  review  of  the  Ad  Mix  Experiment  conducted  for  DOD  by  the 
Wharton  Center  for  Applied  Research  (WCAR)  [3],  The  results  of  this  analysis, 
as  undertaken  by  A.  Charnes,  W.  W.  Cooper,  B.  Golany  and  P.  L.  Brockett  as 
reported  in  [4]  showed  that  (a)  the  statistical  experimental  design  approach 
used  in  the  WCAR  study  was  not  a  suitable  approach  for  addressing  the 
questions  of  how  best  to  budget  for  advertising  in  terms  of  service  specific  and 
joint  advertising  and  (b)  the  models  used  and  the  statistical  analyses  employed 
by  WCAR  were  seriously  deficient. 

Another  part  of  the  Center's  research  effort  resulted  in  a  further  extension 
of  DEA  which  would  more  accurately  reflect  what  is  involved  in  portraying  and 
evaluating  advertising  effort.  This  extension  involved  a  new  type  of  DEA  model 
to  portray  two  stages  of  activity  in  which  outputs  at  one  stage  become  inputs  to  a 
succeeding  stage. 

Figure  3.1  pictorially  portrays  what  is  involved  in  this  two  stage  approach 
in  a  very  simple  way.  In  Phase  I,  on  the  left  of  this  Figure,  inputs  such  as 
advertising  dollars  and  recruiting  efforts  produce  outputs  such  as  "propensities" 
and  "ASVAB  Exams."  These  outputs  are  then  treated  as  inputs  to  a  succeeding 
stage  where,  combined  with  other  inputs  (such  as  recruiting  and  follow-up 
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efforts)  they  result  in  the  contracts  and  other  outputs  that  are  realized  as  shown 
in  Phase  II  of  Figure  3.1. 


Figure  3.1  Two-Phased  DEA 

This  two-phase  approach  differs  from  the  over-simplified  one-phase  only 
approach  used  by  WCAR  in  which  direct  casuality  was  assumed  between 
advertising  expenditures  and  the  production  of  recruits  without  any  intervening 
stages  or  processes  and  without  important  "marketing  variables  such  as 
unemployment  ,  other  service  competition  potential  market  volume,  etc. 
Furthermore,  using  DEA,  a  "production  function"  was  developed  utilizing  FY  84 
recruiting  data  for  each  of  the  Ad  Mix  test  cells  by  aggregating  the  battalion  level 
DEA  results.  In  this  analysis  GSMA  contracts  per  $1000  advertising  was 
utilized  as  a  measure  of  effectiveness.  A  simple  interpretation  of  the  results  is  in 
Figure  3.2: 
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Figure  3.2  Development  of  a  Production  Function 

The  analysis  showed  that  Army  high  quality  contracts  increased  as  the  Army 
share  of  the  total  advertising  budget  increased. [5]  Similar  analyses  for  the 
Marines  and  Navy  revealed  the  same  result:  Service  ads  are  "better"  than  joint 
ads  in  "producing"  high  quality  contracts.  [6,  7]  These  results  conflicted  with  the 
findings  of  the  Department  of  Defense  contractor  (WCAR)  that  actually 
performed  the  Ad  Mix  experiment. 

There  are  other  important  differences  that  should  also  be  noted  between 
the  WCAR  study  and  the  DEA  approach  used  by  the  Center  for  Cybernetic 
Studies.  For  example,  the  WCAR  study  used  data  collected  by  ADIs  (Areas  of 
Dominant  Influence)  whereas  the  DEA  study  used  data  collected  directly  from 
the  battalions  where  recruitment  activity  occurs.  Using  the  latter  approach,  it 
was  possible  to  identify  recruitment  activities  and  possible  aberrations  (e.g., 
inefficient  activity)  with  specifically  identifiable  individual  battalions.  This  was 
not  possible  under  the  WCAR  approach  which  could,  at  best,  only  artificially 
impute  "average  behavior"  back  to  individual  battalions.  Trouble  was  also 
present  in  the  WCAR  data  in  attempts  to  identify  battalion  activities  with  the  ADIs 
with  which  they  were  supposedly  associated. 
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Further  differences  with  the  WCAR  approach  can  be  made  clear  by 
turning  to  Exhibit  3.1  which  shows  a  sample  printout  from  a  particular  battalion 
that  is  readily  obtainable  from  the  DEA  study.  Here,  only  the  Phase  I  inputs  and 
outputs  are  included  but  a  similar  report  can  be  provided  for  Phase  II  or,  if 
desired,  Phases  I  and  II  can  be  combined  into  a  single  report.  To  be  noted  is 
that  multiple  outputs  as  well  as  multiple  inputs  are  included  at  each  stage.  This 
is  in  contrast  with  the  WCAR  study  whose  regressions  had  only  one  output  at  a 
time  as  the  result  of  the  inputs  used. 


SUMMARY  TABLE 

DEA  RUN  TITLE:  ARMY  0335  PHASE  II  54  BNS,  LAGGED  ADVERTISING,  SINGLE  OUTPUT 


DEA  MODEL:  EXTENDED  ADDITIVE 

DECISION  MAKING  UNIT: 

45 

5M 

EFFICIENCY: 

0.883 

REFERENCE  UNITS: 

12 

54 

38 

49 

POTENTIAL 

POTENTIAL 

VALUE  IF 

POTENTIAL 

RATES  OF 

ACTUAL 

EFFICIENT 

IMPROVEMENT 

CHANGE 

OUTPUTS 

CONTRACTS 

243  00 

342  96 

99  96 

1  0000000000 

INPUTS 

ARNAT.AD 

1 48226  70 

114975.16 

33251  54 

-0  0010000000 

JT  NAT.AD 

35291  31 

9694  68 

25596.63 

•0.0010000000 

OS.NATAD 

2  77 

2  77 

0.00 

-143  7318815841 

HSSRPOP 

59147  00 

56189  65 

2957  35 

0.0045420733 

UNEMP85 

9  32 

8  21 

1  11 

-1  0000000000 

PRODRCTR 

107  00 

107  00 

0.00 

-4  9134097931 

Exhibit  3  1  Sample  DEA  Output 

Starting  at  the  top  of  the  sample  output  under  "Summary  Table",  the 
output  provides  a  title  for  remarks  to  assist  in  identification.  Next  "DEA  Model" 
specifies  the  theoretical  version  of  the  DEA  model  in  use.  Currently  there  are 
three  versions  of  the  DEA  formulation,  each  possessing  different  characteristics. 
Choice  of  the  model,  as  stated  previously,  does  not  change  the  earlier  assertion 
that  a  priori  formulations  are  not  necessary.  This  choice  relates  to  the  methods 
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used  to  calculate  the  efficiency  measures  (i.e.  the  ratio  of  the  recruiting  outputs 
to  the  recruiting  inputs).  In  this  sample  output  the  "Extended  Additive"  model 
was  utilized.  This  model  allows  for  extensions  to  distinguish  oetween 
discretionary  and  non-discretionary  inputs  alluded  to  earlier. 

Next  the  Decision  Making  Unit  under  investigation  is  specified.  In  this 
case,  battalion  number  45,  coded  "5M"  is  shown.  This  code  refers  to  the  Peoria, 
Illinois  Recruiting  Battalion. 

The  "efficiency  score"  is  specified  next  on  line  four  at  the  head  of  the 
above  table.  Remember  that  this  score  provides  an  estimate  of  the  efficiency 
utilization  of  inputs  in  "producing"  outputs.  "Reference  Unit,"  on  line  five,  refers 
to  the  collection  of  DMUs  that  were  utilized  by  the  model  in  determining  the 
efficiency  score.  As  the  DEA  model  uses  a  standard  linear  programming  code 
to  solve  the  optimization  problem  that  results  from  the  DEA  formulation,  these 
reference  units  correspond  to  the  "basis"  for  this  solution.  These  efficient  DMUs 
are  "most  like"  the  DMU  under  investigation  in  their  use  of  resources.  Hence 
this  portion  of  the  output  provides  insights  for  comparisons  or  places  to  look  for 
ways  to  gain  improvements  in  efficiency  for  the  DMU  under  investigation. 

The  remainder  of  the  output  shows  the  actual  inputs  and  outputs  used  in 
the  DEA  analysis.  "Actual"  refers  to  the  actual  data  values  that  were  entered  in 
the  data  base  This  repeatback  feature  allows  for  a  rapid  check  of  the  data  for 
each  DMU.  "Potential  Values  If  Efficient"  shows  the  level  of  output  (input)  that 
would  be  produced  (consumed)  if  the  unit  was  operating  efficiently.  "Potential 
Improvement"  is  the  difference  between  "Actual"  and  "Potential"  .  Here  a  wealth 
of  useful  information  is  provided  for  possible  improvements  in  performance.  For 
example,  as  shown  on  the  line  for  contracts,  approximately  100  more  contracts 
could  have  been  gained  with  $33,251  less  national  advertising,  as  shown  on 
the  line  for  ARNAT.AD.  The  specific  amounts  are  not  as  important  as  the 


indication  of  general  managerial  issues  to  investigate  to  improve  efficiency. 

In  addition  to  these  possible  improvements,  the  column  headed 
"Potential  Rates  of  Change"  are  the  rates  of  change  a  DMU  would  experience 
even  after  efficiency  is  achieved  with  incremental  changes  in  that  input  or 
output.  This  rate  of  change  will  be  discussed  in  more  detail  later. 

This  kind  of  printout  is  provided  simultaneously  for  all  the  DMU  specified 
in  the  analysis.  A  simple  input  program  reads  all  necessary  data  at  the 
beginning  of  the  analysis.  The  managerial  implications  of  the  battalion  level 
resolution  are  demonstrated:  decision  support  is  provided  from  the  DEA  as  to 
where  to  focus  resource  allocation  to  effect  changes  in  desired  outputs. 

Still  other  extensions  and  uses  of  DEA  are  available  which  can  be 
explained  from  the  report  in  Exhibit  3.1.  As  shown  in  this  report,  it  is  also 
possible  to  estimate  possible  improvements  that  might  occur  in  the  outputs 
produced  and  the  inputs  utilized  by  this  battalion  if  the  inefficiencies  were 
eliminated.  Note  that  these  inefficiencies  are  identified  and  estimated  for  each 
input  and  each  output.  No  such  identifications  could  be  effected  by  WCAR  from 
which  average  estimates  only  could  be  formed  with  whatever  inefficiencies  or 
confounding  observations  might  be  present  in  the  data  It  should  also  be  noted 
that  the  DEA  kind  of  battalion  information  can  be  aggregated  up  to  ADIs  or  other 
"higher  echelon"  units  (e  g.,  brigades)  as  desired.  Moreover,  such  aggregations 
from  these  DEA  results  can  be  effected  with  efficiency  adjustments,  if  desired, 
from  the  battalions  with  which  they  are  identified.  This  can  be  done  with 
reference  to  different  phases  or  the  two  phases  can  be  combined  without  losing 
the  ability  to  track  possible  inefficiencies  back  to  their  sources  in  the  individual 
battalions. 


4.0  Continued  Research  : 

DEA  -  A  Decision  Support  System  for  Measuring  the  Impact  of 
Advertising 

Research  and  past  experience  have  shown  that  decisions  and 
operations  in  the  Recruiting  Command  require  quantitative  methods  that  are 
sensitive  enough  to  detect  even  the  slightest  variations  in  input/output  intensity. 
Large  scale  experiments  such  as  the  one  attempted  by  WCAR  cannot 
adequately  reflect  the  effects  of  the  attenuation  of  the  inputs  and  outputs, 
particularly  advertising. 

4.1  Purpose  of  Research 

The  purpose  of  this  particular  phase  of  the  research  was  to  further 
explore  the  uses  of  DEA  in  determining  the  impact  of  advertising  in  "producing" 
high  quality  contracts  for  the  U.S.  Army  Recruiting  Command  (USAREC). 
Instrumental  in  this  effort  has  been  the  "team  concept"  in  which  members  of  the 
Center  for  Cybernetics  Studies  and  key  decision  makers  and  project  officers 
have  worked  together  in  all  phases  of  the  modeling  effort  and  analysis. 
Continuation  of  this  concept  is  critical  to  future  successful  research. 

4.2  Data 

The  data  utilized  in  this  current  developmental  stage  comes  from  the 
Defense  Manpower  Data  Center  (DMDC).  DMDC  serves  the  Defense 
Department  as  the  repository  of  all  service  manpower  data  reported  by  the 
services.  DMDC  provided  the  data  collection  and  management  from  the 
original  Ad  Mix  Experiment  discussed  above.  The  Office  of  the  Secretary  of 
Defense  and  the  sen/ices  agreed  that  continued  advertising  data  collection 
would  foster  future  research.  This  secondary  data  provides  a  useful  source  of 


valid  and  increasingly  reliable  data.  The  Center  supports  the  continued 
development  of  this  data  as  the  "industry  standard"  for  military  manpower 
planning  research.  It  should  be  noted  that  DMDC  personnel  have  been 
extremely  cooperative  and  helpful  in  resolving  questions  and  problems  with  the 
data  that  have  arisen  during  the  course  of  this  research. 

Table  4.1  lists  the  major  variables  provided  by  DMDC  (through  USAREC) 
in  July  and  August  1987. 


ARMY  NATIONAL  ADVERTISING 
AIR  FORCE  NATIONAL  ADVERTISING 
MARINE  NATIONAL  ADVERTISING 
NAVY  NATIONAL  ADVERTISING 
JOINT  NATIONAL  ADVERTISING 
ARMY  LOCAL  ADVERTISING 
NAVY  LOCAL  ADVERTISING 
AIR  FORCE  LOCAL  ADVERTISING 
MARINE  LOCAL  ADVERTISING 
ARMY  APPLiCan  l  S 
AIR  FORCE  APPLICANTS 
MARINE  APPLICANTS 
NAVY  APPLICANTS 
POPULATION 


Table  4.1 :  FY  85  Data 


ARMY  GSMA  CONTRACTS 
AIR  FORCE  GSMA  CONTRACTS 
MARINE  GSMA  CONTRACTS 
NAVY  GSMA  CONTRACTS 
ARMY  MISSION 
AIR  FORCE  MISSION 
MARINE  MISSION 
NAVY  MISSION 
ARMY  RECRUITERS 
AIR  FOHCt  RECRUITERS 
MARINE  RECRUITERS 
NAVY  RECRUITERS 
UNEMPLOYMENT 


The  data  were  provided  at  the  county  level  with  battalions  (and  other  service 
equivalent  recruiting  organization)  designators  provided.  DMDC  used  a 
standard  algorithm  approved  by  each  service  to  determine  the  county  to 
battalion  aggregation. 

FY  84  data  were  also  supplied  by  DMDC,  but  as  already  stated  in  [4], 
many  variables  were  only  reported  for  a  portion  of  the  country.  For  any 
temporal  analysis  from  FY84  to  FY85,  the  data  were  augmented  with  official 
USAREC  data  as  necessary. 


Data  reduction  and  preparation  for  analysis  were  performed  on  the 
University  of  Texas  at  Austin  IBM  3081 D. 

Based  upon  guidance  from  USAREC,  the  original  data  were  aggregated 
by  Army  recruiting  battalion  (=DMU)  by  quarter.  Quarterly  observations  were 
chosen,  as  most  recruiting  policies  are  "managed"  by  quarter.  All  variables 
were  summed  from  month  to  quarter  except  recruiters  and  unemployment, 
which  were  averaged  by  quaiter.  Again,  the  team  concept  aforementioned  was 
used  in  determining  inputs,  outputs,  and  DMU's  and  in  any  decisions  regarding 
necessary  data  manipulation. 

This  DMDC  data  allows  for  inclusion  of  competitive  effects  in  the 
analyses  as  all  service  data  is  provided.  Since  each  "management  unit 
designator"  for  each  service  is  provided  on  each  county  level  record, 
aggregation  of  other  service  data  to  Army  organization  was  possible.  This 
service,  provided  by  DMDC,  as  stated  before,  is  vital  to  future  recruiting 
research. 

Numerous  difficulties  were  uncovered  in  the  process  of  "reducing"  the 
data  for  preliminary  analysis.  Specifically,  the  High  School  Senior  population 
had  identical  data  for  all  of  the  approximately  3500  counties  of  the  U.S. 
Additionally,  the  Army  recruiter  data  was  in  error,  showing  less  than  half  the 
proper  number  as  verified  by  USAREC.  DMDC  responded  to  the  Center  for 
Cybernetic  Studies’  requests  for  clarification  and  provided  accurate  data. 
Summary  statistics  were  provided  to  USAREC  in  August  1987  for  verification 
prior  to  preliminary  analysis.  It  should  be  noted  that  these  data  were  well 
documented  (in  most  cases)  and  that  DMDC  was  responsive  to  any  requests  for 
clarification.  The  importance  of  this  resource  to  all  services  for  future  manpower 
research  is  again  highlighted. 


An  initial  subset  of  the  data  was  chosen  for  preliminary  analysis.  This 


subset  (Table  4.2)  was  selected  again  based  upon  discussion  with  UoAREC. 


ARMY  NATIONAL  ADVERTISING 
AIR  FORCE  NATIONAL  ADVERTISING 
MARINE  NATIONAL  ADVERTISING 
NAVY  NATIONAL  ADVERTISING 
JOINT  NATIONAL  ADVERTISING 
MARINE  RECRUITERS 
NAVY  RECRUITERS 
ARMY  RECRUITERS 
AIR  FORCE  RECRUITERS 


ARMY  GSMA  CONTRACTS 
AIR  FORCE  GSMA  CONTRACTS 
MARINE  GSMA  CONTRACTS 
NAVY  GSMA  CONTRACTS 
UNEMPLOYMENT 
POPULATION 

SERVICE  DIRECT  RESPONSE  LEADS 
SERVICE  APPLICANTS 


Table  4.2:  Subset  of  Data  for  Analysis 


4.3  Analysis 

As  a  precursor  to  a  DEA  analysis,  a  typical  regression  approach  was 
utilized  to  gain  insights  into  possible  misinterpretations  of  the  data.  Numerous 
combinations  of  independent  variables  from  the  subset  in  Table  5.2  were 
utilized  in  attempting  to  "explain"  Army  quality  contracts  and  Army  quality 
ASVAB  examination  applicants. 

The  most  revealing  of  these  are  the  simple  regression  lines  for  the  FY85 
data  set  depicted  in  the  following  two  figures: 
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Army  shr  ad 

Figure  4.1:  Regression  of  Army  Applicants  vs  Share  of  Advertising 


Army  shr  ad 

Figure  4.2:  Regression  of  Army  GSMA  Contracts  vs  Share  of  Advertising 

To  be  noted  is  that  both  supplies  are  negative  so  that  in  causal  analysis 
associated  with  these  regressions  it  appears  that  advertising  repels  recruits! 

These  strange  results  are  further  illustrated  in  part  by  the  following 
histogram  of  Army  "share"  of  advertising  versus  Army  "share"  of  applicants. 
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Figure  4.3  Histogram  of  Share  of  Advertising,  Contracts  and  Applicants 

The  bimodal  distribution  depicted  in  Figure  4.3  raises  severe  questions  about 
the  use  of  linear  (or  even  loglinear)  regression  models,  like  those  used  by 
WCAR. 

Regression  models  like  those  used  by  WCAR  are  the  wrong  tools  for  this 
analysis.  The  result  is  a  mis-specified  model  so  that  results  from  this  modeling 
approach  are  best  regarded  as  invalid.  Similar  analyses  of  FY  1984  revealed 
like  results.  In  addition  it  appears  from  the  previously  mentioned  critique  of  the 
WCAR  efforts  that  the  DOD  study  utilized  these  same  techniques  and  reached 
similar  conclusions.  DEA,  in  contrast,  does  not  require  the  kind  of  a  priori 
model  specification  that  these  regression  techniques  require,  so  the  pitfalls 
associated  with  assumption  of  linearity  in  the  relations  to  the  models  can  be 
avoided. 

Previous  uses  of  DEA  in  the  analysis  of  Army  recruiting  have  shown  the 
power  of  the  methodology  as  a  management  tool,  allowing  micro-level 
decisions  at  the  battalion  (=DMU)  level  [5].  Earlier  work  on  the  development  of 


the  production  function  demonstrated  how  a  macro-level  analysis  capability  for 
resource  allocation  could  be  developed  [5].  Finally  the  concept  of  a  micro  level 
DEA  for  each  service  utilizing  DMDC  data  "feeding"  a  macro  level  goal 
programming  model  has  been  conceptualized  in  [8]. 

A  new  development  in  providing  decision  support  in  the  impact  of 
advertising  is  now  presented.  Although  the  general  concept  was  presented  in 
[6],  optimal  dual  variables  can  be  exploited  to  obtain  still  more  from  standard 
DEA  informatics  output.  In  particular  the  rate  of  change  values  shown  for  a 
particular  DMU  as  in  Exhibit  3.1  can  be  used  to  plan  resource  allocations  and 
reallocations  to  obtain  optimum  recruitment  plans  and  strategies  across  all 
DMUs. 

Figure  4.4  shows  a  typical  output  from  a  DEA  analysis  of  the  subset  of 
DMDC  data  described  earlier.  Particular  attention  is  now  called  to  the  values  in 
the  column  headed  "Potential  Rate  of  Change"  where  the  optimal  dual  variables 
are  recorded  . 


Figure  4.4  DEA  Output 


As  shown  in  [1],  the  negative  of  the  ratio  of  this  "potential  rate  of  change"  or 
optimal  dual  variable  for  input  xj  to  the  optimal  dual  for  output  yr  is  equal  to  the 
rate  of  change  of  output  yr  with  respect  to  input  xj.  This  ratio,  then,  provides  the 


Army  with  an  empirical  means  to  determine,  for  example,  the  change  in  quality 
contracts  (output)  with  respect  to  advertising  (input).  Resource  trade-offs  and 
sensitivity  analysis  can  then  be  performed  at  the  battalion  level  or  higher.  Used 
in  this  manner  DEA  provides  marginal  rates  of  substitution  for  the  decision 
maker  [5]  for  each  input  under  observation  by  moving  efficiently  across  the 
frontier  of  possible  solutions.  In  technical  economics  terminology,  these 
represent  marginal  rates  of  substitutions  between  inputs  (or  between  outputs) 
and  marginal  rates  of  transformations  from  inputs  to  outputs  at  the  efficiencies 
frontier. 

4.4  Findings 

The  original  subset  of  FY  85  data  was  aggregated  to  the  Army  battalion 
level  for  all  services.  The  "competitive"  effects  of  "other  service"  (Navy,  Air 
Force,  and  Marine  Corps)  advertising  was  modeled  by  the  following: 

1 

NAVY  AD$  +  MARINES  AD$  +  AIR  FORCE  AD$ 

This  reciprocal  says  that  there  is  an  inverse  relationship  between  "other  service" 
advertising  and  Army  contracts.  (This  relationship  was  developed  in  [5]  and  has 
generally  been  agreed  upon  by  advertising  experts  and  Army  leadership).  The 
"lagged  effect"  of  advertising  was  modeled  by  using  an  average  of  the  previous 
quarter  and  the  current  quarter  advertising  expenditure  data.  Joint  advertising 
was  considered  as  an  input  to  provide  a  "best  case"  estimate  of  its  impact  on 
Army  recruiting. 


To  allow  comparison  to  FY  84,  during  the  period  of  the  Joint  Ad  Mix 
Experiment  in  which  ad  levels  were  changed  in  certain  parts  of  the  country,  the 
FY  84  data  used  in  [5]  were  also  consiaered.  Admittedly,  these  data  were 
provided  in  part  by  USAREC  and  in  part  by  DMDC,  and  so  the  resulting  direct 
comparisons  may  be  biased.  However,  the  DEA  can  control  for  this  in  the  use 
of  a  "window  analysis"  in  which  a  four-quarter  "moving  window"  of  the  same 
DMUs  are  compared  to  other  windows  in  different  time  periods  across  the  eight 
quarters  ,  FY84  to  FY85.  This  window  analysis,  seen  below  in  Exhibit  4.1, 
allows  for  analysis  of  temporal  effects  in  the  rate  of  change  discussed  before. 
Complete  window  analyses  for  the  different  types  of  advertising  (Army,  joint, 
other  service  and  total  service)  are  in  Appendices  A,  B,  C,  D,  respectively. 


100  ■  RATIO  OF  ARMY  DUALS  SUMMARY  MEASURES 


Q1 

02 

CO 

04 

C6 

06 

07 

08 

MEAN 

VAR 

COLUMNTOTAL 

RANGE 

RANGE 

1  A 

0.136 

0.023 

0.393 

0.020 

0  006 

0  086 

0  032 

0.100 

0.115 

0.308 

0.371 

0.465 

0  042 

0.032 

0.032 

0.053 

0.311 

0.049 

0  109 

0.471 

0.100 

0.100 

0.100 

0.100 

MEDIAN 

0.136 

0.014 

0.086 

0.032 

0.074 

0.100 

0.285 

0  100 

18 

0.015 

0.003 

0  008 

0.002 

0  000 

0  006 

0  002 

0.004 

0.013 

0.003 

0.043 

0.045 

0  014 

0  005 

0.007 

0.014 

0.004 

0  047 

0.015 

0.015 

0.047 

0.013 

0.007 

0.011 

MEDIAN 

0015 

0  002 

0  008 

0  003 

0  027 

0.014 

0.011 

0.011 

1C 

0  153 

0  045 

0  100 

0  014 

0  100 

0  100 

0.014 

0  086 

0  092 

0  133 

0.265 

0.311 

0.100 

0.023 

0.282 

0.325 

0  064 

0.017 

0.100 

0.125 

0.020 

0  100 

0.031 

0.040 

MEOIAN 

0  153 

0  072 

0  100 

0.018 

0.053 

0.100 

0.078 

0.040 

ID 

0.152 

0  399 

0  483 

0  940 

0.056 

0  260 

0.013 

0.812 

0.316 

2.626 

1  540 

1  540 

0  100 

0  013 

0  808 

0  235 

1  553 

0.119 

0.367 

0.305 

0.100 

0,230 

0.100 

0.176 

MEDIAN 

0  152 

0  727 

0  260 

0  026 

0.464 

0.235 

0.202 

0.176 

Exhibit  4.1  Window  Analysis 
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Note  that  the  analysis  is  provided  at  the  battalion  (=DMU)  level  with  the  same 
DMU  being  compared  in  different  four-quarter  "windows."  As  statistical 
observations,  these  DMUs  can  be  regarded  as  "different"  in  each  window, 
hence  increasing  the  total  sample  of  units  "observed"  since  the  data  sets  are 
developed  by  dropping  one  quarter  and  adding  another  quarter  of  data  in 
moving  the  window  from  left  to  right. 

Summary  measures  to  the  right  of  the  page  allow  for  rapid  discovery  of 
"exceptions  behavior",  where  a  large  variance  in  measures  may  indicate  outlier 
behavior.  Management  can  then  utilize  this  information  to  direct  attention  to  any 
such  DMU  and  investigate  to  determine  causes  for  this  behavior,  including 
misreporting  or  the  reporting  of  erroneous  data.  Additionally,  median  values 
are  provided  which  allow  for  robust  aggregation  of  the  individual  DMU 
measures  to  national  level  for  macro  analysis.  It  should  be  noted  that  for  this 
temporal  analysis,  all  variables  provided  in  FY  85  were  not  available  in  FY  84, 
so  only  certain  input/output  combinations  could  be  analyzed. 

The  input-output  combination  utilized,  then,  focusing  on  a  single  output 
Phase  II  ("creating  contracts")  DEA  was  as  follows: 

INPUTS  OUTPUTS 

Army  National  Advertising  GSMA  Contracts 

Joint  National  Advertising 

Other  Service  National  Advertising 

HSSR  population 

Production  Recruiters 

Total  Unemployment 
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Phase  II  analysis  and  the  single  output  combination  were  selected  to 
attempt  to  isolate  the  effects  of  advertising  on  contracts.  New  software  has  been 
developed  by  the  Center  for  Cybernetic  Studies  to  provide  the  ratios  of  dual 
variables  for  Army  Advertising,  Joint  Advertising,  Other  Service  Advertising,  and 
Total  Service  Advertising.  Each  battalion  "rate  of  change"  in  contracts  with 
respect  to  each  type  national  advertising  was  summarized  by  use  of  the  median 
rate  of  change  for  each  quarter  in  the  analysis.  These  median  rates  of  change 
were  then  weighted  by  the  battalion  population.  This  weighting  allowed 
aggregation  to  the  national  command  level  by  summing.  Finally  this  command 
rate  was  "averaged"  by  dividing  each  quarterly  command  rate  by  the  total 
population. 

Using  the  previously  described  window  analyses,  comparisons  of  the 
median  rates  of  change  in  contracts  for  each  type  advertising  can  be  made  for 
the  eight  quarters  of  data,  FY  84  -  FY  85.  Results  of  the  analysis  are  depicted 
below  in  Figure  4.5: 


Contracting  Rate  of  GSMA 
per  $  National  Advertising 
by  Quarter* 


Quarter  (FY84-FY85) 

Figure  4.5 

It  is  readily  apparent  that  the  rate  of  change  in  Army  quality  contracts  in 
every  case  is  greater  than  Joint  for  either  Army  advertising,  Other  Service 


22 


advertising,  or  Total  Service  advertising.  Note  in  the  chart  that  the  Army  rate 
declined  drastically  from  the  1st  Quarter  FY  84  to  2nd  Quarter  FY  84.  This  was 
the  beginning  of  the  Joint  Advertising  Mix  Test,  in  which  Service  advertising 
was  either  reduced  or  held  constant,  while  Joint  Advertising  was  raised.  The 
Other  Service  contract  rate  shows  a  decline  in  Quarters  3  through  6,  perhaps 
reflecting  the  effects  of  the  changes  induced  by  the  experiment.  Note 
particularly  the  "recovery  effect"  on  total  services  advertising  with  the 
termination  of  the  Joint  Advertising  Mix  Test  contracting  after  Quarter  6  and  a 
seeming  return  to  "normal".  The  Army  rate  then  remains  relatively  flat,  perhaps 
because  of  the  difficulty  in  recovering  from  the  initial  loss  of  advertising 
awareness.  The  Other  Services  also  show  similar  behavior. 

These  findings  agree  with  earlier  work  performed  by  the  Center  in 
response  to  issues  raised  during  the  Joini  Ad  Mix  Experiment:  Service 
advertising  appears  to  be  more  effective  in  "producing”  contracts  than  Joint 
advertising  [5, 6, 7, 8], 

5.0.  Conclusions  from  DEA  analyses 

Yv 

This  research  is  only  in  the  early  stages  of  development.  DEA  analyses 
need  to  be  performed  with  various  other  service  outputs  and  inputs  to  complete 
these  developments.  However,  the  already  developed  DEA  applications 
provide  an  empirical,  battalion-level  basis  for  management  decisions  regarding 
the  Service-Joint  advertising  issue  and  other  resource  trade-offs.  The  rate  of 
change  measure  is  easily  incorporated  into  previously  developed  informatics 
utilized  for  DEA.  True  decision  support  can  then  be  provided  to  the  recruiting 
command  through  DEA  on  the  impact  of  advertising  of  different  types,  and  on 
the  allocation  of  other  resources.  Thus  DEA  can  provide  the  basis  of  a  Decision 
Support  System  which  will  systematically  provide  insights  from  the  data  while 


maintaining  the  managerial  level  resolution  needed  to  implement  those  insights 


into  decisions. 


6.0.  Future  applications 

As  stated,  more  detailed  DEA  can  be  performed,  utilizing  local 
advertising,  other  service  recruiters,  different  media  types  of  national 
advertising,  direct  response  leads  and  others  to  develop  a  Decision  Support 
System  useful  at  battalion  and  national  headquarters  level.  Insights  can  be 
gained  on  rates  of  change  in  other  outputs,  such  as  changes  in  awareness  with 
respect  to  advertising  inputs  (Phase  I  analysis).  In  addition  the  "efficiency" 
measure  at  the  battalion  level  can  be  utilized  to  assess  changes  in  missioning 
or  sales  quotas.  Finally,  the  micro  level  analysis  can  be  aggregated  to  national 
level  for  each  service  and  “optimal”  resource  levels  can  be  bo  explored  using 
goal  program  extensions  to  DEA  [8]. 

DEA  provides  decision  support  at  the  national  level  producing  a 
quantitative  justification  for  future  executive-level  discussions  on  the  allocation 
of  expensive  recruiting  resources-the  national  advertising  budget.  The  support 
provided,  upon  development,  can  provide  real-time  management  information  in 
highly  relevant  "digestible"  forms  of  reports  for  use  in  the  management  process 
Future  development  will  hopefully  allow  a  successful  integration  of  all  the 
hardware  and  software  into  the  battalion  (and  above)  commander's  decision¬ 
making  environment  so  that  efficienct  resource  allocation  across  brigades 
and/or  battalions  can  be  effected  along  with  monitoring  and  correcting 
inefficiencies  in  the  performance  of  each  DMU  in  the  system 
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Introduction  to  Technical  Appendices 


Appendices  A  through  D  represent  the  window  analyses  described  in 
pages  18-19  of  this  report.  Results  and  conclusions  in  the  report  are  based  in 
part  on  an  aggregation  of  the  median  summary  statistics  for  each  battalion.  The 
entire  window  summaries  are  provided  for  fututer  reference  concerning 
managerial  issues  at  the  battalion  level.  Appendix  A  represents  the  marginal 
rates  of  change  in  GSMA  contracts  for  a  change  in  Army  advertising.  Likewise, 
Appendices  B  through  D  provide  rates  of  change  in  GSMA  contracts  for  small 
changes  in  Joint  advertising,  Other  Service  advertising  and  Total  Service 
advertising,  respectively. 
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